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ABSTRACT 

V 

It  is  shown  that  the  second  order  sufficient  (necessary) 
condition  for  the  dual  of  a  nonlinear  program  is  equivalent  to  the  inverse 
of  the  Hessian  of  the  Lagrangian  being  positive  definite  (semidefini te)  on 
the  normal  cone  to  the  local  primal  constraint  surface.  This  compares  with 
the  Hessian  itself  being  positive  definite  (semidefini te)  on  the  tangent 
cone  on  the  local  primal  constraint  surface  for  the  corresponding  second 
order  condition  for  the  primal  problem.  We  also  show  that  primal  second 
order  sufficiency  (necessity)  and  dual  second  order  necessity  (sufficiency) 
is  essentially  equivalent  to  the  Hessian  of  the  Lagrangian  being  positive 
definite.  This  follows  from  the  following  interesting  linear  algebra 
result:  A  necessary  and  sufficient  condition  for  a  nonsingular  nxn  matrix 
to  be  positive  definite  is  that  for  each  or  some  subspace  of  Rn,  the 
matrix  must  be  positive  definite  on  the  subspace  and  its  inverse  be  positive 
semidefini te  on  the  orthogonal  complement  of  the  subspace. 
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Key  Words:  Nonlinear  programming,  second  order  optimality,  duality 
Work  Unit  Number  5  -  Operations  Research 


Accession  Far 

NTIS  SRAil 
DTI C  TAB 
Unannounced 
Justification- 


¥ 

□ 


By- 


Distribution/ 


Availability  Codes 
Avail  and/or 
Special 


Dist 


Sponsored  by  the  United  States  Army  under  Contract  No.  DAAG29-80-C-QC41 . 
This  material  is  based  on  work  supported  bv  the  National  Science  Foundation 
under  Grants  ENG-7903881  and  MCS-7901066. 


SIGNIFICANCE  AND  EXPLANATION 


Many  constrained  real-world  minimization  problems  are  not  globally 
valley-like.  For  such  problems  we  obtain  conditions  which  tell  us 
whether  we  have  local  valley-like  behavior  both  for  the  original  problem 
and  its  dual.  This  is  useful  in  obtaining  local  upper  and  lower  bounds 
to  the  value  of  the  minimum. 


The  responsibility  for  the  wording  and  views  expressed  in  this  descriptive 
summary  lies  with  MRC,  and  not  with  the  authors  of  this  report. 


LOCAL  DUALITY  OF  NONLINEAR  PROGRAMS 


0.  Fujiwara,  S.-P.  Han  &  0.  L.  Mangasarian 

1 .  Introduction 

We  consider  the  following  nonlinear  program 

minimize  f(x) 

x 

(P)  subject  to  g ( x )  <  0 

h(x)  =  0 

and  its  Wolfe  dual  [14,7] 

maximize  L(x,u,v) 

(x,u,v) 

(D)  subject  to  ?xL(x,u,v)  =  0 

u  >  0 

where  f:Rn-*R,  g:Rn->-Rm  and  h:Rn-»-Rq  are  di fferentiable  functions 
on  Rn,  L(x,u,v):=  f(x)  +  u^g(x)  +  v^h(x)  is  the  standard  Lagrangian 
and  7xL  is  the  gradient  with  respect  to  x.  The  relationships 
between  the  above  two  problems  have  been  extensively  studied  for  the 
convex  case  [14,7].  Our  principal  concern  here  are  local  duality 
results  which  in  the  absence  of  convexity  assumptions  require  the  use 
of  second  order  optimality  conditions. 

In  Section  2  we  give  a  geometrically  meaningful  second  order 
sufficient  optimality  condition  in  Definition  2.1  for  the  dual  problem  (D) 
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and  prove  in  Theorem  2.2  that  it  is  equivalent  to  the  standard  second 
order  sufficient  optimality  condition  [6,2]  applied  to  the  dual  problem  (D). 
Thus,  as  is  well  known,  the  second  order  sufficient  optimality  condition 
for  the  primal  problem  (P)  is  the  positive  definiteness  of  the  Hessian 
of  the  Lagrangian  on  the  tangent  cone  to  the  local  constraint  surface. 

Our  second  order  sufficient  (necessary)  optimality  condition  Theorem  2.2 
(Theorem  2.5)  for  the  dual  problem  (D)  is  that  the  inverse  of  the  Hessian 
of  the  Lagrangian  is  positive  definite  (semidefinite)  on  the  normal  cone 
to  the  primal  local  constraint  surface.  It  is  worthwhile  to  note  that 
while  positive  definiteness  of  the  Hessian  of  the  Lagrangian  ensures  the 
satisfaction  of  the  second  order  sufficiency  condition  for  the  primal 
problem,  this  is  not  the  case  for  the  dual  problem  where  a  constraint 
qualification  is  needed  (Theorem  2.3)  in  order  to  ensure  that  dual  second 
order  sufficiency  holds  under  positive  definiteness  of  the  Hessian  of  the 
Lagrangian. 

In  Section  3  we  characterize  Karush-Kuhn-Tucker  points  of  the  primal 
problem  that  locally  solve  both  the  primal  and  dual  problems  simulta¬ 
neously.  We  show  (Theorems  3.6  and  3.7)  that  these  points  are  essentially 
points  where  the  Hessian  of  the  Lagrangian  with  respect  to  the  primal 
variables  is  positive  definite.  In  order  to  establish  these  results  we 
prove  an  interesting  result  of  linear  algebra  (Theorem  3.5)  which  states 
that  a  necessary  and  sufficient  condition  for  a  nonsingular  n*n  matrix 
to  be  positive  definite  is  that  for  each  or  some  subspace  S  of  Rn,  A 
must  be  positive  definite  on  S  and  A”^  must  be  positive  semidefinite 
on  the  orthogonal  complement  S1  of  S. 
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We  briefly  describe  our  notation  now.  All  vectors  will  be  column 
vectors  unless  transposed  to  a  row  vector  by  the  superscript  T.  For  x 
in  the  n-dimensional  real  Euclidean  space  Rn,  x . ,  i=l,...,n,  will 
denote  its  components.  For  an  mxn  real  matrix  we  shall  say  that 
AeRmxn’  Ai  will  denote  the  ith  row  of  A,  and  if  I<={l,...,m}  then  Aj 
will  denote  the  submatrix  with  rows  A^ ,  id.  For  a  differentiable 
function  g:Rn-*-Rm,  Vg(x)  will  denote  the  transpose  of  the  mxn 
Jacobian  matrix  of  g  at  x.  For  a  twice  differentiable  function 
L:Rn+m-*R,  Vxl(x,u)  will  denote  the  nxl  gradient  with  respect  to 
x,  VuL(x,u)  will  denote  the  mxl  gradient  with  respect  to  u,  V  L(x,u) 
will  denote  the  (n+m)  x  (n+m)  Hessian  with  respect  to  both  x  and  u 
whose  submatrix  components  are  denoted  as  follows 


V2L(x,u) 


7xxL(x’u)  VxuL(x'u) 


7uxL{x’u)  VuuL(x’u) 


A 
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2.  Geometrically  meaningful  second  order  optimality  condition  for  the 
dual  problem 

In  order  to  establish  local  duality  results  without  any  convexity 
assumptions,  second  order  necessary  and  sufficient  optimality  conditions 
become  essential.  The  second  order  sufficient  condition  for  the  primal 
program  (P)  was  given  by  McCormick  and  Fiacco  [6,2]  and  has  been  exten¬ 
sively  studied.  Research  on  this  topic  continues  (see,  for  example,  [4,13]). 
In  this  section  we  formulate  geometrically  meaningful  second  order  necessary 
and  sufficient  optimality  conditions  for  the  dual  problem  (D)  and  study  the 
relationship  to  the  corresponding  conditions  for  the  primal. 

Recall  that  an  (n+m+q)-vector  (x,u,v)  is  said  to  be  a  Karush-Kuhn- 
Tucker  triple  of  the  primal  program  (P)  if  the  following  conditions  hold: 


(a) 

VxL(x,u,v)  =  0 

(b) 

g(x)  <  o 

(c) 

h(*)  =  0 

(d) 

u  >  0 

(e) 

u.gi(x)  *  0 

Such  a  triple  is  said  to  satisfy  the  primal  second  order  sufficient 
optimality  condition  if  f,  g  and  h  are  twice  continuously  differentiable 
at  x  and 

(2.2)  Vgj(x)Td  =  0  ' 

VgK(x)Td  <  0 
Vh  ( x )  Td  =  0 
d  /  0 


d  VxxL(x,u,v)d  >  0 


-5- 


i  ; 


i 


where  J:=  (i|g^(x)  =  0  and  u^.  >  0} 

K:=  {i|g..(x)  =  0  and  u.  =  0}  . 

We  now  give  a  second  order  sufficient  optimality  condition  for  the 
dual  program  (D)  which  we  shall  justify  by  Theorem  2.2  below. 


Definition  2.1  A  Karush-Kuhn-Tucker  triple  (x,u,v)  of  the  primal 
problem  (P)  is  said  to  satisfy  the  dual  second  order  sufficient  optimality 
condi tion  if  f,  g  and  h  are  twice  continuously  differentiable  at  x, 
if  the  Hessian  VxxL(x,u,v)  is  nonsingular  and 


(2.3)  w  =  Vg(x)y  +  Vh(x)z  ) 
i  e  I 
i  e  K 

(y,z)  ^  0 


*i =  0 

Vi  >  0 


wTVxxL(x,D,v)_1w  >  0 


where  I:=  {i|g^(x)<0}  and  K:=  {i|g.(x)  =  0,  u ^  =  0 }  . 

The  geometric  relationship  between  the  primal  and  the  dual  second 
order  sufficient  conditions  is  an  interesting  one.  Let  T  be  a  tangent 
cone  of  the  local  primal  constraint  surface  at  the  point  x  induced  by 
the  second  order  optimality  condition  (2.2),  that  is. 


(2.4)  T:=  {d | Vg j ( x ) Td  =  0,  VgK(x)Td<0,  Vh(x)Td  =  0}  . 


Then  the  polar  cone  of  T,  denoted  by  N  and  called  the  normal  cone  at 
x,  is  given  by 


(2.5)  N:=  Cw|wTd <  0,  V d e  T) 

=  {w|w  =  Vg(x)y  +  Vh(x)z  ,  yj  =  0,  YK>0}  . 
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Therefore,  the  primal  second  order  sufficient  condition  merely  says  that 
the  Hessian  VxxL(x,u,v)  is  positive  definite  on  the  tangent  cone  T, 
while  the  dual  second  order  sufficient  condition  says  that  the  inverse 
Hessian  VxxL(x,u,v)-1  is  positive  definite  on  the  normal  cone  N.  It 
will  be  shown  in  Section  3  that  for  both  conditions  to  hold  it  is  not 
only  sufficient  but  also  necessary  that  the  Hessian  7xxL(x,u,v)  be 
positive  definite  on  the  whole  space  Rn.  We  also  note  here  that  it  was 
proved  in  [4]  that  the  tangent  cone  can  also  be  expressed  as 

T  =  {d jvf (x)Td  =  0,  VgA(x)Td<0,  7h(x)Td  =  0} 

where  A:=  {i | g . (x) »  0}  =  J u  K,  that  is  A  is  the  index  set  of  alj. 
active  inequality  constraints  at  x.  Consequently,  the  normal  cone  N 
can  also  be  written  as 

N  =  {w|w  =  yVf(x)  +  Vg(x)y  +  7h(x)z,  yA^>  0,  Yj  =  0}  . 

These  expressions  contain  the  gradient  Vf(x)  of  the  objective  function 
and  treat  the  index  sets  J  and  K  on  an  equal  footing. 

We  now  justify  Definition  2.1  by  showing  that  the  dual  second  order 
sufficient  condition  given  in  this  definition  is  equivalent  to  the  one 
derived  by  applying  McCormick's  second  order  sufficient  optimality  condi¬ 
tion  directly  to  the  dual  program  (D). 

Theorem  2.2  (Equivalence  of  dual  second  order  sufficient  optimality  condi 
tion  to  McCormick's  condition)  If  (x,u,v)  is  a  Karush-Kuhn-Tucker  tripl 
of  the  primal  program  (P)  and  if  f,  g  and  h  are  twice  continuously  dif¬ 
ferentiable  at  x  then  ( x , u , v )  is  a  Karush-Kuhn-Tucker  point  of  the  dual 


program  (D)  with  the  (n+m)-vector  (0,-g(x))  as  its  Lagrange  multiplier. 
Furthermore,  the  vector  (x,u,v,0,-g(x))  satisfies  McCormick's  second 
order  sufficient  optimality  condition  for  (D)  if  and  only  if  7xxL(x,u,v) 
is  nonsingular  and  condition  (2.3)  holds. 


Proof  The  first  statement  of  the  theorem  follows  immediately  by  direct 
verification.  Notice  that  the  Lagrangian  for  the  dual  program  (D)  is 


given  by 


M(x,u,v,s,t):=  L(x,u,v)  +  sTVxL(x,u,v)  +  tTu 


Let  VM  and  v  M  denote  respectively  the  gradient  and  the  Hessian  of  M 
with  respect  to  (x,u,v)  only.  Thus  we  have 


VM(x,u,v,s,t)  =  p7xL(x,u,v)  +  VxxL(x,u,v)s 

g(x)  +  Vg(x)Ts  +  t 
h(x)  +  Vh(x)^s 


Let  i  =  0  and  t  =  -g(x),  then  it  follows  that 


(2.6) 


V2M(x,u,v,s,t)  =pxxL(x,u,v)  Vg(x)  Vh(x) 

Vg(x)T  0  0 

Vh(x)T  0  0 


Therefore,  McCormick's  second  order  sufficient  optimality  condition  [6,2] 
for  problem  (0)  is  that 


( 1.7 )  VxxL(x,u,v)x  + Vg(x)y  + Vh (x)z  =  0 

y,  •  o 

yKi» 

(x,y,z)  f  0  ; 


(xT,yT,zT)V2M(x,u,v,i,t) 
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where  as  before  the  Hessian  is  with  respect  to  (x,y,v)  only, 

I:=  {i  |  g- (x)  <  0}  and  K:=  {i|g,.(x)  =  0,  =  0}  . 

By  (2.6)  and  the  equality  on  the  left-hand-side  of  (2.7),  the  inequality  on 

T  2 

the  right-hand-side  of  (2.7)  is  equivalent  to  x  V  L(x,u,v)x  >  0.  Hence 
condition  (2.7)  can  be  expressed  as  follows 


(2.8)  VxxL(x,u,v)x  +  Vg(x)y  +  Vh(x)z  =  0 

yj  -  o 

i 0 

(x,y,z)  1  0 


L(x,u,v)x  >  0  . 


We  claim  now  that  condition  (2.8)  implies  the  nonsingularity  of  the 
Hessian  VxxL(x,u,v).  Suppose  it  is  not  true,  then  there  exists  a 
nonzero  x«Rn  such  that  VxxL(x,u,v)x  =  0.  Let  (y,z):=  (0,0),  then 
(x,y,z)  satisfies  the  conditions  in  the  left-hand-side  of  implication 

(2.8),  and  hence  by  implication  (2.8)  we  have  x^VxxL(x,u,v)x  >  0. 

This  however  contradicts  VxxL(x,u,v)x  =  0. 

It  now  follows  from  the  nonsingularity  of  VxxL(x,u,v)  that  the 
condition  (x,u,z)  f  0  in  (2.8)  can  be  replaced  by  (y,z)  f  0,  because 
(y,z)  =  0  implies  x  =  0.  Therefore,  by  defining  w:=  -VxxL(x,u,v)x, 
condition  (2.8)  again  can  be  rewritten  as 


(2.9)  w  =  Vg(x)y  +  Vh(x)z 

yI  "  °  1  ==*  wTVxxL(x,y ,v)_1w  >  0  , 

i  0 

(y ,2)  t  o 
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which  is  condition  (2.3)  of  our  Definition  2.1. 

Conversely,  it  is  obvious  that  condition  (2.9)  and  the  nonsingularity 
of  V^L(x,u,v)  imply  (2.8)  which  is  equivalent  to  McCormick's  condition 

(2.7) .  □ 

It  should  be  remarked  here  that  because  we  used  i  =  0  as  a  multiplier 
for  the  dual  problem  (D)  we  were  able  to  get  away  without  assuming  that 
f,  g  and  h  are  thrice  differentiable  at  x  but  merely  twice  differentiable. 

It  is  important  to  note  that  unlike  the  situation  for  the  primal 
problem,  where  the  second  order  sufficiency  implication  (2.2)  holds  automat¬ 
ically  when  VxxL(x,u,v)  is  positive  definite,  the  second  order  sufficiency 
implication  (2.3)  for  the  dual  problem  need  not  hold  when  VxxL(x,u,v)  is 
positive  definite  because  w  =  0  may  satisfy  the  conditions  of  the  left- 
hand-side  of  implication  (2.3).  However  under  a  slightly  more  stringent 
version  of  the  standard  constraint  qualification  of  nonlinear  programming 

[9.7]  we  can  show  that  positive  definiteness  of  VxxL(x,u,v)  does  indeed 
ensure  the  satisfaction  of  the  dual  second  order  sufficiency  implication 
(2.3)  as  follows. 

Theorem  2.3  (Dual  second  order  sufficiency  under  positive  definiteness  of 
the  Hessian  of  the  Lagrangian)  Let  (x,u,v)  be  a  Karush-Kuhn-Tucker  point 
of  the  primal  problem  (P),  let  f,  g  and  h  be  twice  continuously  differ¬ 
entiable  at  x,  let  VxxL(x,ii,v)  be  positive  definite  and  let  the  following 
primal  constraint  qualification  hold  at  x 
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Vh.j(x)f  i=l,...,q,  Vg^jCx),  are  linearly  independent 
(2.10)  -and  there  exists  a  p  e  Rn  such  that 
VgK(x)Tp<0,  Vg j ( x ) Tp  =  0 ,  Vh(x)Tp  =  0 

Then  the  second  order  sufficiency  implication  (2.3)  holds. 

Proof  Let  (w,y,z)  satisfy  the  conditions  of  the  left-hand-side  of 
implication  (2.3).  If  w  f  0  then  implication  (2.3)  holds  because 
?xxL(x,u,v)”^  is  positive  definite.  We  now  show  that  if  w  =  0  we  con¬ 
tradict  the  constraint  qualification  (2.10).  If  =  0  then  (yj,  z)  f  0 
and  we  contradict  the  linear  independence  of  Vh.(x),  i=l,...,q, 

If  yK  f  0  then  we  have  the  contradiction 

0  =  y^VgK(x)Tp  +  y jVg j ( x ) Tp  +  zTVh(x)Tp  <0.  □ 

Remark  2,4  It  can  be  shown  that  the  constraint  qualification  (2.10) 
implies  the  standard  constraint  qualification  of  nonlinear  programming 
[9  (Definition  11.3.5),  7]  and  (2.10)  itself  is  implied  by  the  often  used 
[12,8]  linear  independence  assumption  of  all  the  active  constraint 
gradients:  Vh.(x),  i=l , . . .  ,q,  Vg.  ft(x) . 

We  now  derive  a  second  order  necessary  optimality  condition  for  the 
dual  problem  which  besides  having  a  geometrically  meaningful  interpretation 
will  be  useful  in  characterizing  simultaneous  local  solutions  of  the  primal 
and  dual  problems.  Recall  that  McCormick's  second  order  necessary  condition 
for  the  primal  problem  (P)  is  that  the  Hessian  VxxL(x,u,v)  be  positive 
semi -defini te  on  the  cone  (d|Vg^(x)^d  *  0,  Vh(x)^d  =  0}  where 
A: =  {i  (g^x)  =  0>.  As  expected,  the  second  order  necessary  condition  for 
the  dual  problem  (D)  is  that  the  inverse  Hessian  7  L(x,u,v)“'*  be  positive 
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semi -definite  on  the  normal  cone  N  defined  in  (2.5).  We  give  this 
result  below.  Note  that,  under  our  assumption  of  nonsingularity  of  the 
Hessian  VxxL(x,u,v),  no  constraint  qualification  is  required  as  is  the 
case  in  McCormick's  second  order  necessary  condition. 


Theorem  2.5  (Dual  second  order  necessity)  If  (x,u,v)  is  a  local 
maximum  point  of  the  dual  program  (D),  if  f,  g  and  h  are  twice  con¬ 
tinuously  differentiable  at  x,  and  if  the  Hessian  VxxL(x,u,v)  is 
nonsingular,  then 


(2.11)  w  =  Vg(x)y  +  Vh(x)z] 


*i 

=  0 

i  e  I 

yi 

>  0 

i  e  K 

wT7xxL(x,u,v)-1 


w  >  0 


where  I:=  {i|gi(x)<0}  and  K:=  {i|g.(x)  =  0  and  0^  =  0}. 

Proof  Let  vectors  w,  y  and  z  be  fixed  vectors  satisfying  the  condi¬ 
tions  of  the  left-handed  side  of  (2.11).  We  consider  the  function 
F:Rn+m+q+1^Rn+ffl+q  defined  by 


F(x,u,v,t):= 


V(x,u,vf 

u  -  u  -  ty 
V  -  V  -  tz 


Clearly,  we  have  F(x,u,v,0)  =  0.  Furthermore,  it  follows  from  the  non¬ 
singularity  of  V  L(x,u,v)  that  the  Jacobian  V  F(x,u,v,0)  is  also 

AA  A , U  j  V 

nonsingular.  Hence,  by  the  implicit  function  theorem,  there  exist  a 
positive  number  e  and  continuously  differentiable  functions  x(t),  u(t) 
and  v ( t )  defined  on  (-e,  e)  such  that  x(0)  =  x,  u ( 0 )  =  u,  v(0)  =  v  and 


-12- 


(2.12)  (a)  7xL(x(t),  u(t),  v(t))  =  0 

(b)  u(t)  =  u  +  ty 

(c)  v(t)  =  v  +  tz  . 

Differentiating  (2.12a)  at  t  =  0,  we  get  that 

7xxL(x,u,v)x'(0)  +  Vg(x)y  +  Vh(x)z  =  0, 
which  imples,  for  w  =  Vg(x)y  +  Vh(x)z,  that 
w  =  -VxxL(x,u,v)x'(0)  . 

Let  0(t):=  L(x(t),  u(t),  v(t)).  Notice  that  for  sufficiently  small 
te[0,e)  the  vector  (x(t),  u(t),  v(t))  is  feasible  to  the  dual  program 
(D)  and  hence  0(0)  >_  0(t)  for  all  sufficiently  small  nonnegative  t. 

On  the  other  hand,  we  have  that 

o'  (0)  =  Vxl(x,u,v)Tx'(0)  +  g(x)Ty  +  h(x)Tz  =  0. 

Therefore  it  follows  that  0"(O)  <  0.  By  direct  verification,  we  have  that 

9" (0)  =  -wTVxxL(x,ii,v)-1w  . 

Hence,  the  proof  is  complete.  □ 
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3.  Characterization  of  simultaneous  local  solutions  of  the  primal  and 


dual  problems 

In  this  section  we  characterize  Karush-Kuhn-Tucker  points  of  the 
primal  problem  that  locally  solve  both  the  primal  and  dual  programs 
simultaneously.  We  will  show  that  these  points  are  essentially  those 
Karush-Kuhn-Tucker  points  at  which  the  Hessian  of  the  Lagrangian  with 
respect  to  the  primal  variables  is  positive  definite.  To  establish  this 
we  need  a  preliminary  fundamental  result.  Theorem  3.5  below,  which  appears 
to  be  an  interesting  linear  algebra  result  in  its  own  right.  Related 
results  have  appeared  in  [10].  First  we  recall  a  well  known  result. 

Lemma  3.1  (Sherman-Morrison-Woodbury  formula  [11])  Let  A e  Rnxn  be 


nonsingular  and  let  U,  V«R 


Then  A  +  UV  is  nonsingular  if  and 


only  if  I  +  V  A”  U  is  nonsingular,  in  which  case 


(A+UV7)"1  =  A-1  -  A-Vl+vVM-VA-1  . 


We  can  now  state  and  prove  our  preliminary  fundamental  result. 

Theorem  3.2  (Characterization  of  positive  definitness  of  a  nonsingular 
matrix)  A  necessary  and  sufficient  condition  for  a  nonsingular  matrix 
A  e  Rn  n  to  be  positive  definite  is  that  for  some  or  each  a  >  0  and 
8  e  Rm*n ,  the  matrices  A  +  aB7B  and  I  +  aBA^B7  be  positive  definite. 

Proof  (Necessity)  Let  A  be  positive  definite,  then  obviously  for  each 
<x  >  0  and  each  BeRmxn,  the  matrices  A  +  aB7B  and  I  +  aBA'^87  are 
positive  definite. 
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(Sufficiency)  Let  A  be  nonsingular  and  let,  for  some  a  >  0  and 
some  Be  Rm+n,  the  matrices  A  +  aB^B  and  I  +  ctBA’^B^  be  positive 
definite.  By  the  Sherman-Morrison-Woodbury  formula  we  have  that 

A'1  =  (A+ctB^r1  +  aA‘1BT(I+aBA‘1BT)‘1BA‘1 

which  is  the  sum  of  a  positive  definite  matrix  and  a  positive  semidefinite 
matrix  and  hence  A'1  and  consequently  A  is  positive  definite.  □ 

By  noting  that  if  I  +  aBA'^B^  is  positive  definite  for  all  a  >  a 
for  some  a  then  BA’^B^  must  be  positive  semidefinite  the  following 
corollary  follows  from  Theorem  3.2. 

Corollary  3.3  (Alternate  characterization  of  positive  definiteness  of  a 
nonsingular  matrix)  Let  A  e  RnXn  be  nonsingular.  A  is  positive 
definite  if  and  only  for  all  a  >  a  for  some  a>0  and  for  some  or  all 
B€Rmxn,  A  +  aBTB  is  positive  definite  and  BA_1BT  is  positive 
semidefinite. 

A  very  nice  geometric  version  of  Corollary  3.3  can  be  stated  if  we 
make  use  of  the  following  well  known  result  of  positive  definitness  of 
quadratic  forms  on  a  linear  subspace. 

Lemma  3.4  (Finsler-Debreu  Lemma  [3,1])  Let  A  e  Rnxn  and  let  BeRmxn. 
Then 


/tk  +  aB^B  is  positiveN 
definite  for  all  a  >. 
vfor  some  a  >  0 


r 


Combining  this  lemma  with  Corollary  3.5  gives 

Theorem  3,5  (Geometric  characterization  of  positive  definiteness  of  a 
nonsingular  matrix)  The  nonsingular  matrix  A  in  Rn*n  is  positive 
definite  if  and  only  if  it  is  positive  definite  on  some  or  each  subspace 
of  Rn 

S  =  {x|Bx  =  0},  where  BeRm*n, 

and  A'1  is  positive  semidefinite  on  the  orthogonal  complement  S1  of  S: 

Sx  =  (y  |y  =  BTu}  . 

We  remark  here  that  the  matrix  A  may  not  even  be  positive  semi¬ 
definite  when  A  is  positive  definite  on  both  the  space  S  and  its 
orthogonal  complement  S1.  This  can  be  seen  from  the  example: 

A:=  f|.  S:=  {x£R2|x2  =  0}  . 

We  are  now  ready  to  present  our  characterization  results. 

Theorem  3.6  (Positive  definiteness  of  Hessian  of  Lagrangian  under  primal 
second  order  sufficiency  (necessity)  and  dual  second  order  necessity 
(sufficiency))  Let  (x,u,v)  be  a  Karush-Kuhn-Tucker  triple  of  the 
primal  program  (P)  that  satisfies  the  strict  complementarity  condition: 
u1  >  0  whenever  g^(x)  =  0,  and  let  the  Hessian  Vxxl(x,u,v)  be  non¬ 
singular.  If  (x,u,v)  satisfies  the  primal  second  order  sufficient 
(necessary)  optimality  condition  and  the  dual  second  order  necessary 
(sufficient)  optimality  condition,  then  VxxL(x,u,v)  is  positive  definite. 
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Proof  Notice  that  when  the  strict  complementarity  condition  holds  K 
is  empty  and  the  tangent  cone  T  defined  in  Section  2  is  a  subspace 
and  the  normal  cone  N  is  its  orthogonal  complement.  When  the  primal 
second  order  sufficient  (necessary)  condition  is  satisfied  the  Hessian 
VxxL(x,u,v)  is  positive  definite  (semi definite)  on  T.  While,  when  the 
dual  second  order  necessary  (sufficient)  condition  holds,  the  inverse 
Hessian  VxxL(x,u,v}_1  is  positive  semidefinite  (definite)  on  N. 
Therefore,  it  follows  from  Theorem  3.5  that  the  Hessian  VxxL(x,u,v) 
must  be  positive  definite.  □ 

In  light  of  the  above  theorem,  it  is  natural  to  expect  that  the 
Hessian  would  be  positive  semidefinite  when  both  the  primal  and  the  dual 
second  order  necessary  conditions  hold.  Curiously  this  turns  out  not  to 
be  true  as  can  be  seen  from  the  following  example: 


min  -x.j  x^ 


s.t. 


x.|  =  0 


-T 


The  vector  x  =  (0,1)  together  with  v  =  1  constitute  a  Karush-Kuhn- 
Tucker  point.  Both  the  Hessian  of  the  Lagrangian  and  its  inverse  are 

[.?  -g  •  -  also  have  the  tangent  space  T  =  (x£R2|x1  =  0}  and  the 

normal  space  N  =  {xeR  |x2*0}.  Therefore,  both  the  primal  and  the  dual 


second  order  necessary  conditions  hold.  But  the  matrix 
positive  semidefinite. 


r°  -ii  i 

-1  oj 


s  not 


The  positive  definiteness  of  the  Hessian  VxxL(x,u,v)  clearly 
implies  the  primal  second  order  sufficient  condition.  By  Theorem  2.3 


under  the  constraint  qualification  (2.10),  the  positive  definitness  of 
VxxL(x,u,v)  also  implies  the  dual  second  order  sufficient  optimality 
condition.  Therefore,  we  have  a  converse  to  Theorem  3.6  which  extends 
and  sharpens  Luenberger's  local  duality  result  [5], 

Theorem  3,7  (Primal  and  dual  second  order  sufficiency  under  positive 
definiteness  of  Hessian  of  Lagrangian  and  constraint  qualification)  Let 
x  be  a  local  minimum  point  of  (P)  satisfying  the  constraint  qualification 
(2.10),  let  f,  g  and  h  be  twice  continuously  differentiable  at  x  and 
let  (u,v)  be  a  Lagrange  multiplier  associated  with  x.  If  VxxL(x,u,v) 
is  positive  definite  then  (x,u,v)  satisfies  both  the  primal  and  the  dual 
second  order  sufficient  optimality  conditions. 
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